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Abstract: The present paper is based on two empirical research studies. The Netbook 1:1 project (2009-2012), funded by 
the municipality of Gentofte and Microsoft Denmark, is complete, while Students' digital production and students as 
learning designers (2013-2015), funded by the Danish Ministry of Education, is ongoing. Both projects concern primary and 
lower secondary school and focus on learning design frameworks that involve students' agency and participation regarding 
digital production in different subjects and cross-disciplinary projects. Within these teacher-designed frameworks, the 
students perform as learning designers of learning objects aimed at other students. Netbook 1:1 has shown that digital and 
multimodal production especially facilitates student-learning processes and qualifies student-learning results when 
executed within a teacher-designed framework, which provides space for and empowers students' agency as learning 
designers. Moreover, the positive impact increases when students as learning designers participate in formative evaluation 
practices. Traditionally, the Danish school has worked hard to teach students to verbalise their own academic 
competencies. However, as our everyday environment becomes increasingly complex with digital and multimodal 
technologies, formative evaluation as a learning practice becomes central, requiring the students to develop a digital and 
multimodal literacy beyond the traditional, language-centred type. In order to clarify these practices, we address the 
various understandings of evaluation and assessment that may blur our arguments. Students' digital production and 
students as learning designers is a large-scale project that follows up on the findings of Netbook 1:1. It experiments further 
with various evaluation practices in a digitalised learning environment that focuses on different phases of the learning 
processes and includes feed-forward and feedback processes. Evaluation as a learning practice in a digitalised learning 
context focuses on students as actors, addressing their self-reflections, responses to feedback from peers and feed¬ 
forward processes, and responses to feedback from teachers and feed-forward processes. We find that apart from teacher 
initiated and planned evaluations, the teachers find it useful to initiate ad-hoc evaluations in order to grab interesting 
aspects on the fly. At the same time we see students initiate ad-hoc peer-evaluations and make appointments for 
swopping their work for peer-evaluation. 

Keywords: Assessment, evaluation, formative, summative, self-evaluation, peer evaluation, teacher evaluation, digital 
learning processes, multimodality, evaluation design, agency, empowerment, reflection, construction of meaning 


1. Introduction 

Traditionally, Danish schools have worked hard to teach students to verbalise their own academic 
competencies. However, the advent of new media impacts on the representative forms of subject matters and 
calls for new subject knowledge. The representative forms are specifically changed through multimodality; the 
subsequent, new subject knowledge is needed not only because the technology is available, and multimodal 
competencies are core competencies of the 21 st century (Organisation for Economic Co-operation and 
Development [OECD] 2008). New subject knowledge is also necessary because multimodal means of 
expression function as an externalising vehicle for reflection and construction of meaning (Sprensen & 
Levinsen 2014), in the same way as speech and writing externalise thoughts and enhance learning (Dysthe, 
Hertzberg & Hoel 2001). With multimodal expressions, the students achieve a broader repertoire through 
which to experience the world and express themselves while learning. Gradually, as education becomes more 
digitalised and multimodal, schools are challenged to provide new subject knowledge, together with 
professional capacities, to describe multimodal competencies and specify learning objectives and what must 
be taught and evaluated. 

Since the 1990s, especially in the UK, there has been a growing interest in developing formative evaluation, 
since evaluation has proven to be a powerful learning practice in contemporary constructivist learning designs, 
such as cooperative, action or problem-based learning that aim at students' active participation and agency 
(Black & Wiliam 1998, Harlen & Deakin Crick 2002, Tanner & Jones 2003, Hattie & Timperley 2007). At the 
same time, formative evaluation functions as a means of navigation for the actors during the process of 
transformation undertaken in schools as digitalisation expands (Luckin et al. 2012). This development has led 
to an increasing interest in digitally supported, formative evaluation practices, a field marked by new ideas and 
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innovation, such as experiments with mobile technologies and social networks (Ibid.). Summative evaluation 
has recently been actualised by the Programme for International Student Assessment (PISA) that ranks 
students' performance among the OECD member-countries. The PISA results and the ranking of countries have 
set the agenda for educational politicians, educators and researchers across Europe, shifted the focus from 
formative to summative evaluation (Dysthe 2009, Shewbridge et al. 2011, p. 7) and released huge investments 
in digital, multiple-choice tests and measuring designs (called The Digital National Tests in Denmark). In both 
formative and summative digital evaluation designs, multimodality and students' agency stand weak (Wyatt- 
Smith & Kimber 2009)- Accordingly, the development and production of evaluation designs that encompass 
both constructivist and social constructivist, digitalised learning designs or learning designs that involve use of 
digital resources, and the various modalities and their interrelations as representations, emerge as a major 
challenge. Moreover, since their definition 50 years ago, the concepts of formative and summative evaluations 
together with the distinction between evaluation and assessment have become weakly defined (Taras 2008). 

Our research in Netbook 1:1 (for further reading about research design, data collection and analysis, see 
Levinsen & Sprensen 2013 and Sprensen & Levinsen 2014) shows that various designs of evaluation formats, 
ranging from students' self-evaluation over peer evaluation to teacher-initiated evaluations, are crucial for 
students' subject-related and trans-disciplinary learning, the way they work and how they design and conduct 
presentations. In the ongoing research and development project, Students' digital production and students as 
learning designers (2013-2015), we experiment with various evaluation practices, learning objectives and 
criteria for evaluation. The present paper describes the basic framework for formative and summative 
evaluations and various evaluation formats that have been developed and tested in the project, illustrated by 
empirical examples. Additionally, we briefly touch up on theoretical approaches and new challenges, such as 
student governance and multimodality in evaluations. 

2. Literature study 

The teacher's awareness of the students' knowledge (or lack of it) and abilities is pivotal to the design of a 
learning framework that involves students' agency. In practice, to facilitate learning, both the teacher and the 
student must achieve awareness of the student's (pre)condition and progress (Luckin et al, 2012, p. 40). To 
achieve this awareness and transform it into a teacher's assessment and the students' reflected understanding 
of their learning levels and challenges, the generally agreed practice is evaluation/assessment - a huge topic 
ranging from national tests to the kind of everyday classroom practices that are the focus of the present 
project. These practices are divided into two main categories: formative and summative (Shriven 1967, Bloom, 
Hastings & Madaus 1971). 

The research literature regarding the respective roles of assessment and summative and formative evaluations 
in students' learning and empowerment as agents in relation to their learning is primarily produced in the UK. 
According to Tanner and Jones (2003), Taras (2008) and Black (2013), the research focus has mostly been 
directed towards formative rather than summative approaches. Taras pointed out that the relationship 
between the two forms is weakly described: "More seriously, the absence of this discussion has resulted in the 
distortion of this relationship, which has led to a misunderstanding of both assessment processes" (Ibid. p. 
173). 

Taras (2008) and Tanner and Jones (2003) argued that the formative and summative concepts are often used 
to denote and distinguish between two types of evaluation: teacher and classroom assessment (formative 
assessment) versus external/national assessment (summative assessment). In contrast to the general 
definition, Brookhart (2001) emphasised function and purpose rather than type and claimed that both 
classroom and external assessments are applicable to formative and summative purposes. Tanner and Jones 
(2003, p. 276) defined an "assessment event" to include: "...the preparation for the assessment by both the 
teacher and the student, the feedback from the assessment offered by the teacher and the impact of the 
assessment on the subseguent learning behaviours of the student". Accordingly, the formative purpose of any 
evaluation or assessment should be to provide feedback that helps students identify gaps and assist in their 
learning progress and performance. 

The works of Black and Wiliam (1998, 2009) and Hattie and Timperley (2007) showed that positive impact on 
students' learning performance is strongly connected to interaction and continuous evaluation by the teacher 
as an integrated part of everyday teaching. Luckin and colleagues (2012) found that assessment as a learning 
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practice, combined with digital production, strongly supports students' academic and cross-disciplinary 
learning, the way they work and how they shape their projects and presentations, while their awareness of 
their own capabilities changes substantially as they progress. In the UK, the so-called assessment-for-learning 
movement had promoted formative over summative approaches until recently (Taras 2008); the consequence 
has been a research focus locked on the negative effects of testing (Black 2013) leaving the term evaluation to 
be currently more preferred than assessment (Taras 2008). As a counter movement, the present PISA 
programme promotes summative evaluation as a driver for learning over formative evaluation (Dysthe 2009). 
In this situation, most research aims at producing evidence that either of the types performs better than the 
other, rather than focus on how they might enhance each other or how they should be understood or defined 
(Taras 2008). 

Consequently, the meaning of the terms evaluation and assessment and their relation to summative and 
formative purposes have become quite ambiguous and weak defined. When perceived as belonging solely to 
the teachers' domain, they are often understood as interchangeable. However, due to the scientific dispute 
mentioned above the term evaluation is currently preferred over assessment (Taras 2008). In contrast Luckin 
and colleagues in their NESTA report Decoding Learning: The Proof, Promise and Potential of Digital Education 
(2012) relate the concept of evaluation to the researchers studies of learning practices. Assessment is related 
to teacher domain and teacher initiated informal or formalized measures, while reflection refers to student 
involving, formative activities that belong to both the students' and the teachers' domains. In North America 
the concepts are often understood opposite to the European tradition (Matyi 2010) which may also be a 
source of confusion. 

3. Formative and summative evaluations as learning practices 

In the following we adopt Lucking and colleagues distinction between the concepts assessment and reflection. 
However we also use the term evaluation as a dialogue, inquiry and agency driven learning practice that 
produces reflection, feedback and feed-forward for both teacher and students. 

The general research focus has been on the teacher-driven evaluation and assessment that aim to provide 
both teacher and students with knowledge that improves the students' performance, as presented in OECD 
Reviews of Evaluation and Assessment in Education - DENMARK (Shewbridge et al. 2011). The focus on the 
teacher as the main provider of feedback and feed-forward corresponds with the general view within the 
learning design theory that learning design - including evaluation and assessment - is the teacher's domain 
(Dale 1989, 2000). In contrast to the generally accepted position, we discovered in our research (Levinsen & 
Sprensen 2013, Sprensen & Levinsen 2014) that even young students are able to act as learning designers of 
digital learning resources aimed at other students. They were able to reflect on both their learning and 
working process, and the learning design and esthetical qualities of their own and others digital products. We 
found that practices ranging from students' self-evaluation over peer-evaluation to teacher-driven formative 
evaluations in various designs are crucial for students' subject-related and trans-disciplinary learning, the way 
they work and how they design and conduct presentations. The students actively reflect on and use both the 
feedback and feed-forward produced in both ongoing evaluations during the work process and final 
evaluations. We also found that the students' ability to act as learning designers depended on the teachers' 
ability to design and facilitate a frame that supported the students. 

In the ongoing project, Students' digital production and students as learning designers, we take these ideas a 
step further. We coin the concept of students as learning designers, meaning that students act as learning 
designers of their own learning processes, including forms of evaluations within a teacher-designed framework 
that empower students' agency through digital production of learning objects aimed at other students. In the 
project, we experiment with various evaluation practices with formative purposes, learning objectives and 
criteria regarding evaluations that: 

• focus on different phases of the learning processes 

• contain feed-forward and feedback processes 

• focus on students as actors and address their self-reflection, responses to feedback from peers and feed¬ 
forward processes, and responses to feedback from teachers and feed-forward processes 
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As mentioned above, Luckin and colleagues (2012, p. 40) pointed out two important processes - reflection and 
assessment - in order to identify what and how the learner knows and understands. The first is the student's 
reflection on his or her own learning activities. In doing so, the student develops an understanding and 
becomes empowered about how to improve his or her learning activities and strategies. These reflections are 
qualified through dialogue, inquiry and practice. The second process is the teacher's professionally based 
assessment of and reflection on the student's learning activities. The learning processes are qualified by the 
teacher's feedback and feed-forward to the student, as the student should learn to respond to critical 
comments and include them in the ongoing learning activities and reflections. In our understanding of 
students as learning designers, students' reflection and teachers' ongoing assessment interact through a 
shared evaluation practice within the teachers' learning design framework. This framework (for further 
reading see Sprensen & Levinsen 2014) provides a basis for ongoing formative and summative evaluations 
where the evaluation activities may function as a driver for reflection and learning for both teacher and 
student. 

In agreement with Brookhart (2001), we understand summative and formative evaluations as functions that 
may take place both inside or outside the (hybrid) classroom and with the purpose of providing both teacher 
and students with appropriate knowledge to strengthen and stimulate students' academic development. Along 
with Scriven (1967) and Bloom and colleagues (1971), we understand these functions as having different 
points of departure. Thus, summative evaluation lends itself to a backward-looking perspective that aims at 
evaluating completed tasks and whether (minimum) competencies have been achieved in relation to defined 
learning objectives. On the other hand, formative evaluation refers to a forward-looking perspective that aims 
at producing feed-forward that supports the students' future learning, ways of working and collaborating, and 
presentations and presenting performance. Thus, we understand evaluations as learning practices that are 
performed by both teacher and students as functions with a purpose. Summative evaluation is mostly 
conducted at the end of a learning process and may include assessment and measuring, while formative 
evaluation may take place both during and at the end of a learning process. The two evaluation practices are 
complementary; summative information is about the distance to the goal (learning objective), while formative 
information helps define and navigate the journey (learning) towards that goal. 

As mentioned above, we find that design for learning is both the teacher's and the students' domain. Netbook 
1:1 (Sprensen & Levinsen 2014) showed that the students operate at all three levels of Dale's model (1989, 
2000): practice, organising and planning and theory, due to a teacher-designed frame, which centres on the 
students' learning process and subject-related reflections. The frame defines an arena that invites and 
motivates the students to take on responsibilities and act as learning designers under the teacher's 
supervision. We find that students evaluate and reflect on their practice during these processes by using 
everyday language, while the teacher's reflection in action is based on his or her professional theoretical 
knowledge and competencies. 

From a design-for-learning perspective where the students' learning is driven by their reflections and 
evaluations as learning designers, we have further developed Dale's model into a four-level Design for 
Learning Model. In the new model (Figure 1), the fourth level including formalized assessment belongs solely 
to the teacher's domain. 


1. Practice 

Teacher 

Student 

2. Organization and planning 

Teacher 

Student 

3. Situated and practice-based reflection 

Teacher 

Student 

4. Theory-based reflection 

Teacher 



Figure 1: Four-level design for learning model 

Taking the departure point from the students as learning designers, we argue that evaluation becomes a 
pivotal practice for both the learning process and learning results and that evaluation should be a core learning 
practice in any teacher-based design frame, explicitly formulated in terms of function and purpose, and 
actively integrated into the students' agency as learning designers. Apart from Luckin and colleagues' (2012) 
two processes, we work with three variations of evaluation practices: 

• students' self-reflection and self-evaluation 

• peer response with feedback and feed-forward 
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• teacher response with feedback and feed-forward 

Before we unfold the evaluation practices, we should explain the feedback and feed-forward concepts that are 
present in all three practices and form the basis for evaluation as a learning practice. Feedback aims to: 

• take stock 

• share knowledge 

• evaluate a process and/or a result 

Feedback is a practice where summative and formative purposes may function complementarily. When 
feedback aims at taking stock, it is often given during the learning process. However, feedback may also 
function as summative at the end of a learning process and provide an overview to the student about what is 
or is not learned and which competencies are acquired or not. When the aim is sharing constructed knowledge 
among students, feedback may equally be generated during and/or after the process. When the aim is to offer 
a critical or an appreciative response, evaluation is typically performed at the end of the process, when the 
work is completed and presented. Feed-forward aims to: 

• proceed with a process 

• generate new ideas 

• produce changes 

Sometimes, feedback and feed-forward are intertwined, since feedback often leads to conversations that raise 
questions and reflections about how to proceed, and knowledge sharing leads to new ideas. However, in our 
context, it is convenient to focus on feed-forward if the purpose is solely formative, such as if the students are 
at a standstill, the teacher finds a lack of subject-matter quality in their work, the organisation of their work 
does not function, and changes are needed. Based on assessment, the teacher may choose to intervene and 
challenge the students to reset their usual ways of thinking and doing and inspire new strategies to proceed. 
Feed-forward may also occur spontaneously, for instance, if a student finds useful ways of using a program or 
an app or discovers information that may be shared with the class. As feedback, feed-forward may be given 
both during and after a learning process. For both teacher and students, feedback and feed-forward may 
function as drivers for redesigning both the frame and the learning design. 

4. Research design and methodology 

The project is based on a combination of Action Research and Design Based Research using quantitative and 
qualitative approaches. The overall framework for the project includes interventions within different subjects. 
The interventions are in accordance with Design Based Research designed with increasing complexity from 
simple mathematic exercises to more complex trans-disciplinary activities that involve advanced technologies 
such as social media, robotics or location-based technologies. In accordance with Action Research, the 
researchers and the teachers collaborate closely when preparing the interventions locally at each school as the 
interventions must be integrated in the ordinary planning of the school year. 

The project is complex in many ways and produce data using two main approaches within an overall mixed 
methods framework (for details see Levinsen et al. 2014): 1). Baseline measures are conducted as a long term 
diachronic quantitative survey combined with qualitative structured observations at the start, middle and end 
of the project; 2) each of the six interventions are followed through a combined synchronic and diachronic 
approach where the researchers as action researchers follow the interventions in order to document and 
identify changes and developments of the performed practice. Qualitative data are collected before, during 
and after the interventions in the form of individual semi-structured interviews, semi-structured focus groups 
and informal conversations with teachers and students, as well as video, photo and artefacts. The aim is to 
produce a complementary set of data that records and documents the interventions and allows for analysis of 
their impact on the students' learning and the teachers practice. 
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5. Students as learning designers, and evaluation practices 

In this section, we present examples of both teacher initiated and student driven evaluation practices from the 
projects Students' digital production and students as learning designers and Netbook 1:1, based on subject- 
related, trans-disciplinary and individual contemplative projects. 

5.1 Students' self-reflection and self-evaluation 

The teacher develops a digital objective and an evaluation sheet for the students to fill in at the end of a 
project. The evaluation sheet is included in the student portfolio and used during student-teacher and student- 
parent-teacher conversations. In the early grades, the students possess limited writing skills, so the sheet has a 
mix of check marking and a few written answers. The teachers plan the activity with a generous time frame 
and perform formative guidance while the students evaluate themselves in relation to the following 
categories: Subject/trans-disciplinary learning; Working practice - organisation; Collaboration; Product and 
presentation; Technology; Modalities; and Working efforts. 

In this manner, the students' evaluation refers to the teacher-defined objectives for the project, which stem 
from the legislated learning objectives for Danish schools. These objectives are discussed with the students for 
their understanding and rephrased to student-language. Figure 2 shows examples regarding Sepp's working 
practice and Alberta's understanding of the subject matter. 


Do you mostly talk or listen when you 
collaborate with others? 

Sepp 

1 mostly talk but 1 also listen pretty much 

What is special about Ole Lund Kirkebys 
way of writing? 

Alberte 

He writes many things and he writes something 1 can 
understand and something that explains and means 
something 


Figure 2: Example of a digital questionnaire in which second-grade students answer questions after studying 
an author and his books (authors' translation) 

In the early grades or if the students are not yet used to evaluation as a learning practice, it is important to 
introduce the format thoroughly, since its purpose is to provide an arena for the students' reflections on their 
learning process and the learning itself. 

It may be an issue of whether check marking supports reflection. Flowever, in the early grades, it is important 
that all categories are present and allow the students to become accustomed to and build a repertoire of 
relevant criteria to master the evaluation of various learning objectives and the dimensions of the learning 
process. 

5.2 Multiple representations and functions of learning objectives 

In Danish as native language teaching, 1 st grade remediates a known fairy-tale into a multimodal 
representation with 5 th grade as the intended target group. One general learning objective is that the students 
learn to negotiate content and means of expression in groups while the specific context addresses learning 
objectives such as: play and experiment with language; genre; multimodality; and digital production 
(Undervisningsministeriet (Danish Ministry of Education) 2009). 

First the teacher facilitates a class session using the Interactive Whiteboard, where the students suggests 
requirements that secure that 5th grade will appreciate the remediated stories (Figure 3). These requirements 
end up fulfilling the learning objectives regarding genre. 


Students requirement 

Subject matter category 

Others can tell who is who 

Characters 

Others can see how they feel 

Emotions 

Others can see where they are 

The place 

Others can understand what's going on 

Narrative structure 


Figure 3: Learning goals turned into student-language requirements 
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The groups work out a paper based storyboard with hand drawn sketches (Figure 4) where they turn the 
requirements into a criteria for evaluation: Can others (5 th grade) understand what's going on? 



Figure 4: Fland drawn storyboard 

This leads to changes in the order of the sketches as well as details in the sketches, in some cases initiated by 
the students and in others by the teacher. The groups decide the roles (actors, instructors, camera, set 
manager etc.) and start to produce their storyboard using an iPad (Figure 5). 



Figure 5: Mise-en-scene: Flanse and Gretel's poor starving family in the cold and miserable cottage; instructor 
and photographer evaluate the takes. 

Now the students use the requirements as criteria to frame the production using questions as Can others (5 th 
grade) tell who is who and where they are? and Can others (5 th grade) see how they feel? The students were 
able to work on the production for more than half an hour without teacher intervention. In the following 
teacher initiated class session, the groups display their work using wireless access to the Interactive 
Whiteboard. The requirements now frame the peer suggestions for improvements such as: If you take a close- 
up of Flans holding the stick ... Then we understand what's going on. The peer-session is followed by a new 
iteration where the productions are finished. For the final presentation the class has - facilitated by the 
teacher - transformed the requirements into a set of summative questions to 5 th grade that test the quality of 
their work, e.g.: Can you see who that is? Flow is Flans feeling? Why does he feel that way? ... Can you repeat 
the story? The response from 5 th grade leads to discussions regarding Red Riding Flood: Can a boy act as the 
grandmother and a girl as the hunter? and the suggestion from a 1 st grade girl: if they have something that 
shows grandmother or hunter, then you can tell who is who. 

5.3 Responses with feedback and feed-forward 

Ongoing evaluations with feedback and/or feed-forward can be used as short time-outs, where students 
and/or the teacher show and tell something that others can learn from, for example, when students have 
found out how to animate a graphic element. A complementary summative and formative purpose may also 
be included at the end of a learning process. 
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Teachers practice in the class 

R- [T IT ft [T R R 


Students prepare 
production 

Students production 
Ongoing evaluation 

Students evaluate 
production/learning 





->Time 

Figure 6: Ongoing evaluations with feedback (backward-looking critiques and comments) and/or feed-forward 
(forward-looking comments and suggestions on what can be done) 

In our project, the students continuously uploaded their work or collaborated directly in the cloud. Therefore, 
all products, notes and stages in the processes were accessible for both the teacher and the peers at all times. 
They were both easily shared online and mediated using the interactive whiteboard during time-outs. 

Peer response: During the process, peer response is provided as planned or as spontaneous time-outs where 
the students take stock of the progress in relation to the overall time frame. Short presentations using the 
interactive whiteboard allow for peer comments on the quality of the work and suggestions for improvements. 
The whiteboard can be used as a tool to add layers and comments directly on the digital work. As an 
evaluation practice with a formative purpose, peer response is both useful for subjects and trans-disciplines, 
working practice, and use of technologies and modalities, since it supports ongoing improvements. Peer 
response as a final evaluation of subjects and trans-disciplinary projects focuses on the students' digital 
products, the acquired subject or trans-disciplinary learning, and the working practice, including subject- 
related and social collaborative aspects. Peer response as a final evaluation of individual contemplative 
projects focuses on each student's final product and the way the student uses modalities to present and 
mediate the product and the content. During the peer response, the teacher stays in the background and 
guides the evaluation in terms of how to express relevant critiques without hurting the receiver and ensuring 
that all relevant aspects are brought into play. The students' self-evaluation is also an element in the final peer 
response as it is a driver for the formative function of peer response. 

10th. grade students work with digital production. Digital production entails working with creative writing 
based on text and picture stories they have read a week before. The students use the program inklewriter to 
which the teacher gives a brief introduction. The students choose individual characters from the story and 
convert a branch structure based on the selected characters. The branch structure means that the students as 
authors of the various stories will develop two possible endings for the story; e.g. a good and a bad ending. 
The program inklewriter has an integrated structure that provides this kind of narrative. The teacher 
introduces the construction of branch structure and present an example of a student produced branch story as 
inspiration. Students work in groups of two. 

The production process is designed so that it entails ongoing teacher initiated peer evaluations. At the start of 
the second module the teacher provides a short presentation to peer response where the students must read, 
review and comment on each other's texts in opponent groups. First, the students have to read each other's 
stories and then give response regarding language, narrative comprehensibility and genre in relation to the 
original short story. A peer response could for example entail encouragement from fellow students regarding 
writing more nuanced and not use repetitive terminology or sentence types. "You have to use a thesaurus" is 
the appeal from one group to another - and an online thesaurus is shown. The students examined immediately 
expressed great satisfaction with the process of peer response. Later in the same module the opponent groups 
will change so different groups will evaluate each other, which may be initiated by the students themselves. 
The observation showed that peer responses were both initiated by the teacher as well as by the students 
themselves. 

The final evaluation showed that both teachers and students were very excited about working with branch 
stories in Inklewriter. Both sides found that it was an inspiring and creative way of working. The teacher found 
that the students have written a lot and with a good quality. In addition, the students found that their stories 
were largely qualified by the ongoing peer response, as the following quotes from students' shows: "We 
learned a lot of peer evaluation", "peer evaluation is the best form of evaluation" "good to be a response group 
of other students," "you get inspiration from other groups" and "we provide other good ideas". 
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Teacher response: The teacher may both produce process and final evaluations and hold summarising 
conversations with individual students. As the learning manager, the teacher needs to maintain awareness of 
the various aspects of the learning processes and the students' challenges, then uses this knowledge to select 
important issues for either process or final evaluations. The teacher's professional background and knowledge 
about the students help him or her select what to allocate to the plenum and individual conversations, 
respectively. The evaluation may be organised in various ways; accordingly, the teacher must possess a 
repertoire of practices that suits the evaluation purposes, such as: Are there only right or wrong answers or 
alternative possibilities? Is the purpose to reflect on a subject or to inspire more in-depth work with the 
subject matter? What kind of evaluation enhances the students' future learning processes, and what may be 
vulnerable? 


For a full week, 10 th -grade students work on trans-disciplinary projects, using art and architecture in their 
neighbourhood. They employ various digital production tools to produce short videos, texts, photos, etc., 
about the neighbourhood. These are published through the social location-based, mobile gaming platform 
SCVNGR that allows users to build a game-layer on top of the world. During the week that also includes peer- 
response sessions; the teacher accesses the digital productions and takes stock of the students' process, 
progress and challenges. This knowledge is used to provide formative input to the students in the form of 
feed-forward regarding the thoroughness of their mediating content and the functional and aesthetic use of 
their modalities. The teacher will for instance select a few productions based on which he gives feed-forward 
by a dialogical form with the students. Based on this feed-forward, the teacher and the students together set 
specific criteria for the quality of the productions, which subsequently function as guidelines for the following 
work with the students' productions. At the end of the process, the final file constitutes the students' delivery 
of the assignment for the teacher's assessment. 

6. Concluding discussion 

Although the project Students' digital production and students as learning designers (2013-2015) is not 
finished and we are only able to present initial analyses, we find evidence that the interaction between 
student-formulated requirements/learning objectives and the various forms of formative evaluations strongly 
impact on the students subject learning and their ability to perform qualified learning design and evaluations 
of their own and others work - both regarding subject matter and esthetical qualities. We find that the 
teachers change their everyday practice as they experience the learning impact and begin to implement class 
dialogues about learning objectives and formative evaluation practices in their learning design. We also find 
that the students voluntarily initiate formative evaluations and use the requirements actively while the 
student-formulated concepts are gradually replaced with the subject matter concepts. However, the study also 
opens for a new range of research interests, of which a few are mentioned in this concluding section of the 
paper. 

The use of language bears an impact on whether the students take the evaluation as a formative input for 
future work. Littleton and colleagues (2005) developed the concept of "thinking together", addressing various 
forms of dialogue and questioning that may either enhance or hamper collaboration and learning. "Thinking 
together", combined with an explorative manipulation of digital representations, is also useful in evaluation 
processes, especially when language functions as a vehicle for exploration and construction of knowledge and 
as a driver for reflection. The underlying reason is that the teacher may use the students' articulation of and 
work with the digital product to identify challenges and exploit the acquired knowledge to reorganise the 
frame or details in the learning design. In the Danish school, similar to higher education, no tradition exists for 
evaluating the teacher's work.. Nonetheless, we note an advantage in involving students in this aspect of the 
teacher's practice, for instance, the learning design and the teacher's communication and interaction with the 
students that may point at future research interests 

Digital technology offers a wide range of options for developing evaluation as a learning practice. We found 
that teachers and students used shared networks and cloud technology to create new dynamic frameworks for 
both the teachers' work with evaluations and for the students' peer evaluation. Accordingly, there is a need 
for research and development regarding how technology may support summative and formative evaluation 
practices during and after learning processes. It is also necessary to explore and develop digital solutions that 
enhance the analysis of evaluation results and support the teachers and students decision making on how to 
transform produced knowledge into efficient practices. Teachers are already exploring digital options; Luckin 
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and colleagues (2012) specifically cited social media. E.g. the use of audio-visual technologies and text based 
questionnaires involve a range of modalities that expands the need for literacy beyond written texts and 
spoken words. The very notion of both formative evaluation and classroom assessment is also challenged by 
the hybrid space that emerges through mobile and wireless technology and dissolves the brick-and-mortar- 
defined limits of the physical classroom (Luckin et al 2012, de Souza e Silva 2006, Sprensen, Audon & Levinsen 
2010 ). 

We have already mentioned the beginning impact of digitalisation on the forms of assessment such as digital, 
national summative types and the competitive approach used in the PISA programme. However, other 
dimensions such as multimodality, creativity and innovation that are actualised by digitalisation also influence 
assessment. When students' approach to subjects become multimodal, and subjects are mediated digitally 
through multimodal representations, multimodality becomes part of both the subject matter itself and the 
related academic competencies. As mentioned in the introduction, the Danish school has aimed at teaching 
students to verbalise their own academic competencies. However, the advent of new media and their impact 
on representative forms and competencies call for new subject knowledge, not only because the technology is 
available and multimodal competencies represent a core competence in the 21 st century (OECD 2008). New 
subject knowledge is also needed because multimodal modes of expression function as an externalising 
vehicle for reflection and construction of meaning, together with speech and writing (Sprensen & Levinsen 
2014). Thus, multimodal means of expression provide students with both a broader repertoire for expressing 
themselves and a more nuanced tool for experiencing the world. This means that multimodal means of 
expression not only have to be part of the articulation of evaluations and assessments in a digital learning 
environment, but the students' use of the former should be assessed as a competence with defined learning 
objectives. This issue is an important dimension in future discourses on any subject and its related knowledge 
regime. For the teachers, it is a major challenge to develop learning designs for evaluation/assessment-as-a- 
learning practice in a digitalised learning environment and to include both the modalities and their 
interrelations. 
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